Emotionally mediated crossmodal correspondences affect classification performance

Crossmodal music-to-color correspondences are mediated by emotion for classical music (Palmer et al., 2013) and diverse other genres. People tend to choose colors as going best with music when both have similar emotional associations (e.g., happy-looking colors go best with happy-sounding music). Lower level musical stimuli, including single-line melodies, two-note intervals, and instrumental timbres show analogous emotional effects, as do music-to-texture associations (Peterson et al., VSS-2014). Other crossmodal correspondences without emotional mediation (e.g., size/pitch associations) affect classification performance, modulate motion perception, and influence multisensory integration/perception (Spence, 2011). Do emotionally mediated crossmodal correspondences also produce such effects? We find that people are slower and/or less accurate at classifying the emotionality of stimuli (e.g., saturated yellow as happy) when simultaneously presented auditory stimuli are emotionally incongruent (e.g., the sad sound of a clarinet or minor chord) than when it is congruent (e.g., the happy sound of a piano or major chord).